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THE POLITICS AND PROCESS OF EVALUATION AMD DECISION MAKING IN 
THE SCHOOLS: THE SHAWNEE MISSION, KANSAS, EXPERIENCE 

A Paper Presented to the 
Regional Conference 
of the 

National Council for the Social Studies 
Virginia Beach, Virginia 
April 24-26, 19^5 

By Robert Crane 
National Assessment of Educational Progress 

School administrators, faced with the trend toward increased accounta^ 
bility and aware of the delicate relationship that often exists between 
the politics and process of evaluation, have been hard pressed to develop 
sound program evaluation models acceptable to school boards, teachers, 
parents and students. The administration of the Shawnee Mission (Kansas) 
schools, however was able to solve an explosive political and educational 
issue by use of test instruments developed by the National Assessment of 
Educational Progress (NAEP) . ^ 

For many years Shawnee Mission has been regarded as one of the top 
school districts in the country^ It is a predominantly upper middle-class 
school district, drawing its students from the affluent suburbs of Kansas 
City, Missouri. These communities identify strongly with quality education 
and have always been willing to pay for it* The school district presently 
has a $46 million budget. Two-thirds of its students go on to college. 



^National Assessment> federally funded by the National Center for Education 
Stat5<stics, gathers information on edu^,at lonal outcomes in 10 subject- 
matter areas. It develops, administ€^r$, analyzes and reports its findings 
on a continual basis. NAEP questions are based on stated educational 
objectives and can be used one at a time* Approximately one-half of the 
questions asked in each assessment become public domain and can be used by 
state and local school systems in assessing their own programs. 



are always well represented among recipients of national academic achieve- 



ment awards and score well on the college entrance exams. 

During the i972'-73 school year, a task force of 30 social studies 
teachers was set up to make recommendations for the eighth- through twelfths- 
grade social studies curriculum. Based on the changing emphasis of local, 
state and national goals for social studies instruction and on the findings 
of a randomly conducted survey of teachers, students, parents and adminis- 
trators, the task force proposed establishing a new program called American 
Studies. This course was to be offered as an alternative to the required 
American History and American Government courses and would be applied 
toward the graduation requirement for social studies. 

The American Studies curriculum centered around five recurring themes 
in the American experience* These included; how man develops his identity, 
growth patterns of groups, what the process of change looks like, what 
constitutes a significant influence and how we see ourselves as a result* 



Using a thematic modular approach, rather than the conventional chrono-- 
logical approach, the teachers who developed American Studies hoped to 
accomplish several things* "The thematic approach Is not necessarily the 
only way, nor even a new way to teach," commented one teacher, "but the use 
of themes gets back to developing a thinking process; too often teachers 
are the only ones to ask questions, and this shortchanges students*" 
According to another developer, "The primary advantage is the chance to 
do more individualized work than we usually do* Teachers usually teach to 
the middle students; with this approach we can use our materials to reach 
all the kids/' Another stated, "If the emphasis is too much on fact 
retention, what you get is sequential forgetting. A student can cram and 
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memorise to pass exams but he soon forgets i£ the facts have no significance 
to him. This process does not give him a continued interest in learning/' 
Furthermore, modular units added flexibility to the curriculum. They 
could be added or removed from the themes on a continuing basis as the 
needs o£ students dictated. 

The new program was outlined for the board of education and the five 
high school advisory boards in May 1973. Initially the plan called for a 
pilot program in two high schools that had volunteered to take part. 
Although an evaluation of the program was envisioned, it had not been 
planned or initiated. The assumption, however, was that the criterion** 
referenced tests, created to measure the goals and objectives of American 
Studies, would be used to evaluate the new program. 

The proposed American Studies program became the center of a heated 
controversy at the August 1973 school board meeting. School board opponents 
based their objections to the program on four arguments. 

* The program would be used to indoctrinate students with radical ideas. 

* The program '^as unstructured and lacked supervision. 

• The program was untested and would use students as guinea pigs. 

• Students enrolled in the program would not learn certain essential 
facts of American history* 

On the issu^ of indoctrination, a leading spokesman for the critics 
felt schools had no business teaching attitudes, that it was the responsi- 
bility and right of parents. "There's just too much room in the program 
for teachers to teach their own philosophy* I consider it an attempt to 
change the student's way of thinking. When they're messing with the 
thinking of my child, I resent it." 



ERIC 
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As a decision-making body, the school board had never interfered in 
curriculum issues before. However, in this instance, the critics on the 
school board *had prepared arguments against the program that could only be 
answered by some hard facts which were not available: Were Shawnee Mission 
students being used as guinea pigs as had been suggested? Would students 
in the new program learn fewer facts about Ainericaa history? _Could the 
new program stand up against the traditional courses which showed proven 
results? The administration was now placed on the defensive. Although 
four of the seven board members supported the new program, they found them-^ 
selves In a difficult position. In order to continue their support^ they 
demanded answers and exerted promises from the administration for a 
thorough evaluation of the program. Based on these promises, the school 
board upheld the American Studies program as a pilot by a narrow four*to- 
three vote. 



The Shawnee Mission administration ?ound itself under the gun after 
the August 1973 school board meeting. They were faced with two options: 

1. The administration could capitulate to the conservative faction 
of the school board which wanted the American Studies program 
discontinued before it began. 

2. The administration could satisfy the school board demand that the 
new program be evaluated against the existing program. Given the 
promise of an objective evaluation, the pilot would be allowed to 
continue through the 1973-74 school year. However, models for 



Under the Gun 



implementing step-by-step evaluations in input/output terms for 



new programs; were not to be found. 




The first option seemed untenable to the administration. They found 
it intolerable to have classroom policy dictated by a school board. This 
had never happened before in Shawnee Mission, and both teachers and admin- 
istrators feared the precedent that could be established. Dr. ktzel Ball, 
the Superintendent of schools who supported the new program, put it quite 
bluntly in a newspaper interview. "We used the Planning and Development 
Dep'^rtment to survive," he said. "Without the evaluation I probably 
wouldn't be here now/' 

Given the lack of "real" options, the first attempts to develop an 
evaluation instrument began immediately* The task of developing a fair, 
objective evaluation that would convince both the board of education and 
the classroom teachers who would have to participate was delegated to the 
Planning and Development Department by Superintendent Ball. The key element 
of the evaluation from the board's point of view had to be a factual know- 
ledge survey that would be an unbiased measure of changes in achievement 
levels.in both the traditional program and the new program. Without it 
the new program would be scuttled. The teachers and evaluation team, cn 
the other hand, had a broader view of how program effectiveness might be 
measured. Steps were taken to address their concerns as well. 

Although Shawnee Mission hao introduced new programs before, the stakes 
were never very high. As the director of the evaluation observed, '*In the 
past, the curriculum specialists kept evaluations under their control and 
piloting a new program was always a perfunctory task* Everyone paid lip 
service to the idea> but it really meant th^it the program was already 
accepted. We never compared a new program to ongoing one — that w^is too 
threatening. " 




6 



The evaluarirn of Americaxi Studies was breaking new ground, and there 
were a variety of issues that made the task very different: 

1* The superintendent had put his job on the line by supporting the 
new program. There was a clear desire to vindicate his position. 
2, The school board had set precedent by becoming Involved in here- 
tofore internal school policy. They had to be convinced the 

evaluation was reliable and not a cover^-up. " 

3* The natural fears of the teachers that they, not the programs, were 
being compared, and that a new program could be forced upon them, 
had to be overcome. 
4* The evaluation was coming from the administration rather than the 

teachers themselves. 
The plan called for Chree output measures: (1) a factual achievement 
test, (2) a survey of study habits and attitudes and (3) a student opinion 
grid. These measures approached the evaluation from three distinct vantage 
points in the attempt to get at some of the intangibles that are difficult 
to measure, but nonetheless important aspects of a student's total learning 
experience. 

The decision to use a well*-known survey of study habits and attitudes^ 
and to develop the Student Opinion Grid themselves was made rather easily 
and quickly. The development of the Factual Knowledge Survey, around which 
the ultimate success or failure of the evaluation hinged, presented greater 



^SSHA Survey of Study Habits and Attitudes, Form H. Grades 7-12. Copy- 
right 1953, 1964, 1967 by The Psychological Corporation, 304 E- 45th Street, 
New York. 




difficulty and proceeded with greater caution. In order to make the 
factual survey credible both to the teachers and .the school board, an 
impartial source of social studies test materials had to be found* While 
the teachers in both programs could agree that there was a core of basic 
social studies knowledge that al.l the students should have* they could not 
find a test that was suitable* Most norm-referenced standardized tests 
have to be used in entirety; however, the Shawnee Mission evaluators had 
more specific requirements in mind. Although they wanted a core of agreed- 
upon items» they also wanted to include items that teachers in both programs 
could submit to alleviate any fear that the evaluation was stacked for or 
against a particular group. 

Shawnee Mission extended feelers to National Assessment as a source 
for independent, credible questions to evaluate the program. In late 
August, the director of the evaluation, Robert Cramer, inquired about using 
HA£P social studies exercises for the Shawnee Mission evaluation. Informed 
that HAEF could and would provide the assistance he needed, Shawnee Mission 
received a set of NAEF social studies questions for leview by the end of 
August and had arranged to confer with HAEF^s assessment specialists in mid- 
September. Aft^.r the September meetings, the decision was made to cse the 
National Assessjient items as the core of neutral questions in the Shawnee 
Mission factual survey. Having located an acceptable source of social 
studies questions that could also be tailored to Sha'.mee Mission^s specialized 
needs, the Planning and Development staff now felt they had the basis for 
a factual knowledge survey.^ Further stigg^ctions and the general testing 

^The factual knowledge survey used by Shawnee Mission can be found in Appen^ 
dix A. The National Assessment items are marked with an asterisk 
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expertise of NAEP put them in firtn control of the evaluation. Looking 
back, the director commented, "They were fresh items that had credibility 
in the field. NAEP served as the final catalyst to get the evaluation off 
the ground." 

The three outcome measures were administered as both pre-* and posttests. 
The Factual Survey and the Study Habits and Attitude Survey were given 
twice; the Studlent Opinion Grid, three timeis during the year. All stu-^ 
dents enrolled in American Studies, American History or American Government, 
approximately 1,000 in all, were tested each time so that no one student 
or teacher felt singled out. The test booklets were carefully monitored 
and had to be checked in and out by the department chairman each time 
tttey were used. 

A summary of the year-^long study was presented to the board of educa-* 
tion at the end of June 1974. The evidence showed that students did not 
learn fewer facts in a thematic approach to American history, in addition, 
students in the thematic approach scored significantly better in terms of 
etudy habits and had move positive attitudes toward social studies. A 
district spokesman felt this was the most important finding of the evalua-* 
tion. "In this course,*' he said, "we were not only trying to provide kids 
with a basic knowledge of American history and government, but also to 
teach students the tools and resources available to them to acquire more 
knowledge* '* The associate super in tende^f^^or planning and evaluation 
found the results of the classroom monitoring very impressive. "A 
criticism of this course was that it would allow students to do whatever 
they wanted,** he said. "The fact is that a course of this nature calls 
for a higher degree of planning by the teacher and as a result, a higher 
degree of structure.** 



Stressing that the evaluation clearly showed that students In American 
Studies learned the facts as well as students in traditional courses, the 
superintendent made the following recommendation to the board of education: 



This recommendation was adopted without dissent. 

In the narrow administration sense, the Shawnee Mission evaluation 
effected a very Important change. The American Studies program Is now a 
part of the curriculum and must be offered at all five senior high schools 
as an alternative program. Had the evaluation shown that students In 
American Studies learned fever facts than those in the traditional program, 
the pilot program would not have been recommended for continuation by the 
superintendent . 

Teachers benefitted from the experience as well. For many the concept 
of "evaluation'' has* always been anathema, and, at the onset, they felt their 
worse fears would be realized. The professional manner In which th 
evaluation was handled went a long way toward changing this attitude. The 
most basic fears of teachers^that their teaching styles would be regimented 
or that a program would be forced on them^ — did not come to pass. In fact, 
the evaluation has tendeJ to broaden the options rather than narrow them. 
Furthermore, the experiment In social studies has conditioned other depart- 
ments to accept the concept of evaluation more open},y. The language arts 
teachers are presently working with the Department of Planning and Evalua- 
tion to initiate a study of rhelr own- 



II. 



The administration recommends, on the basis of this 



evaluation, that the American Studies program b> con- 



tinued In the district as an option for students.'^ 
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The students are clear winners. As a result of the evaluation they 
now have a more flexible curriculum with increased options for learning. 
Based on a recent tabulation of course enrollmeiit by the direci^or of social 
studies, it appears that general enrollment in social studies courses is 
up this year. If in fact this trend, suggesting greater interest, continues, 
the evaluation will have proved successful beyond the furthest hopes of 
the administration. What is evident, in any case, is that everyone is more 
comfortable with this social studies program in Shawnee Mission, Kansas, 
these days. * 



I2> 
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APPENDIX A 

FACTUAL KNOWLEDGE SURVEY 
AMERICAN STUDIES 



Instructions to the student: The following survey is designed to be com- 
pleted in forty*five (45) minutes. It has two parts: Part I: Short 
Answer — listing — allow 35 minutes to coaq[>Iete; Part II: (/rite on one 
discussion topic—allow 10 minutes to complete. When you complete Part I» 
go right on to Part II. 



PART I* Short Answer - 35 minutes 

^1. A major American manufacturing corporation seeks to establish a branch 
plant in a country that has rich natural ^resources but very little 
industry. The leaders of the nation turn down the American corpora* 
tfon*s request. What reasons can you give for the decision made by 
the leaders of the foreign nation? 

1. 



2. 



3. 



*2k Which one of^the'foUowing events led DIRECTLY to the United States* 
declar^ation of war in the Second Iforld War? 

a. The sinking of the Lusitania 

b# The German occupation of France 

c. The Japanese attack on Pearl Harbor 

The signing of a secret Nazi^Soviet pact 

e. I don*t know. 



The asterisk (^) denotes National Assessment test ltems> 
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As we look at what man has done to his environment and vhat he is 
doing at the present to his enviromeent^ it is apparent that: 

a. t^He bas had a great ileal of foresi ght and wisdom in the ways 

he has modified his aurroundlngs. 
b. The physical setting has had little effect on his economic 

and political decisions. 
c. He has paid little attention to the totality of the effects 

of the environnent on the quality of life. 
d. He will be able to solve his encvironinental problems as they 

relate to the quality of life in the next 30 years or so. 



Which of these will not be a XToblem for our government in the next 
30 years or so? 

^ a. To make economic decisions based on possible zero population 
growth, increased life expectancy and further reduction of 
farm population 

b. To make economic and political decisions based on the 

uncertainties of population growth and worldwide political 
and economic stability 

c. To make political and economic decisions based on the possibl 

loss of freedom due to Increased surveilance sophistication 
and the feasibility of data banks 

d. To make political and economic decisions based on the cer* 

tainty that with the help of ever advanced computers* the 
best political and economic system will be perfected 



Which of these factors that account for the rise of isms in various 
countries is out of place? 

a. Comiminism in Ru^si'^-^laie jarfiwal of the industrial revolu- 
tion and the ia^bility^>f the Christ government to deal 
with its problems 

b. Communism in China— inability of the Chiang government to 

deal with the problem of land distribution 

c. Capitalism in the United States — determination of the Ameri- 
cans to develop an economic system completely different from 
that of Western Europe 

d. Comomnism in Cuba — great inequalities of wealth between the 

very rich and the very poor 

List 3 examples of "cultural universals." 
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2. 



3t 



Identify five American politicians on any level of government whose 
efforts and/or ideas have significantly affected the lives of other 
human beings: 



1. 
2. 



3. 
4. 
5. 



List five individuals and groups vhose efforts* ideas and inventions 
have significantly affected the lives of other human beings and 
describe their contributions: 

1. 



2. 



3. 



^^1 



IC 
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9. Who delivered the ''Cross of Gold Speech'^? 

a. William Jennings Bryan 

b. Walt Whitman 

c. Mark Hanna 

d. Theodore Roosevelt 



10. Which period in American History has been called the "Era of Flaming 
Youth/' the ^'Lawless Decade^" and the "Jazz Age"? 



a. 
b. 
c. 
d. 



the 1890s 
the 1930s 
the 1920s 
the 1950s 



11. American writers who exposed undesirable fe<itures of business at the 
beginning of the twentieth century were called: 

a. Populists 

b. muckrakers 

c. scalawags 

d. of romanticists 



12. Which of the following statements is false? 

a. Fascism is a political system of the extrencf right. 

b. The United States government is a good example of a direct 

democracy. 

c. A good conservative will defend the status quo against major 

changes in the political » economic or social institutions 
of his society. 

d. A theocracy is a political system in which political power is 

exercised directly or indirectly by a clergy and in which 
church law is superior to or replaces civil law. 



13. Which of the following was not a Reform of the Good Years 1900-1914? 

a. Child Labor Legislation 

b. Pure Food and Drug Legislation 

c. Anti^Trust Actions 

d. Social Security 



lERiC 



Members of Congress are most likely to see and hear lobbyists (and 
their arguments): 

a. at social gatherings. 
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b. at luncheon engagements. 

c. at committee hearings. 

d. in their offices and hotel rooms or apartments- 



^15. Name as many reasons as you can why laws are needed: 



*16. A. What do you think are three important problems generally facing 
large cities in the United States? 

1. 



2. 



3. 



B. Which one of the problems you named would you most like to ask 
questions about? 

Problem 1. (Go to C) 

Problem 2. (Go to C) 

Problem 3. (Go to C) 

None (Go to next exercise.) 

C. What two questions would you like to ask about this problem to 
find out more about it? 

1. 



2. 
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*I7. A. What are the two largest political parties in the United States? 
1. 

2. 

B. Are there any other political parties in the United States? 

yes 

no 



C. (If yes) Can you name one other? 

yes 

no 

D* What is it? 



*I8. The American Declaration of Independence states: 

"We hold these truths to be self-evident » that all men are 
created equals that they are endowed by their Creator with 
certain unalienable Rights^ that among these are Life» 
Liberty and the pursuit of Happiness~That to secure these 
rights. Governments are instituted among Men, deriving 
their just powers from the consent of the governed**," 

Do the following sentences restate ideas expressed in this quotation 
from the Declaration of Independence? 

fi* Some rights can never he rightfully taken away. 

yes 

no . 

I don^ t know. 



B* Governments get their right to govern from the people, 

yes 

no 

I don^t know. 



*19* When Columbus sailed across the Atlantic, would he have been able to 
do the following things at about that time in history? 
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A. Would he have been able to send a telegraph inessage? 

yes 

^ no 

B. would he have been able to determine directions with a compass? 

^ yes 

^ no 

C. Would he have been able to read a printed book? 

_^ yes 
_ no 

D. Would he have been able to travel fifty cjiles an hour? 

yes 

no 



*20. Which one of the following has the power to declare an Act of Congress 
unconst i tu t ional ? 

the Congress 

the President 

the United States Supreme Courts' 

The United States Department of Justice 

I don't know. 



21. Read th . following statements made by four speakers. You will be 
asked to tell which speaker bases his opinion on facts rather than 
on his personal feelings. 

Speaker I: I think Hunter lost the election .because metobers of his 
own political par^^y failed to support him. I heard 
from Walters that the Crash Party decided at a secret 
meeting not to work for Hunter but to support his 
opponent, Pallor. 

Speaker II: I don't think so. Hunter's campa'^n manager told me 
that Pallor's workers were buying votes. They were 
offering the people two dollars each if they would 
vote for Pallor. That's why Hunter lost. 

Speaker III: I disagree with both of you. According to a poll 
taken^^after the election, 84 percent of the people 
questioned said they voted for Pallor because he 
promised to lower taxes. Hunter lost the election 
because of this issue. 
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Speaker 17; It is perfectly obvious that Hunter lost because of 
his religion. iVe heard i^eople all over town 
whispering that they would never vote for a member of 
that religion. 

Which speaker bases his opinion more on fa'^ts than on his -personal 
feelings? 



Speaker I 
Speaker II 
Speaker III 
Speaker IV 
I don*t know. 



*22. Suppose the teacher has asked all of the students in your class to 
find out how many people immigrated to this country in 1900. Which 
one of the following do you think would be the MOST dependable or 
reliable source for finding out how many people imnigrated to the 
United States in 1900? 

An article about immigrants in a popular magazine 

A very old neighbor v;ho Immigrated to the United States 

A fiction story about a young girl who came to this country and 

who became a citizen 
Immigration dates and figures in a book put out by the United 

States Census Bureat 

I don* t know. 



Part II: Choose any one (1) of the following discussion topics. Allow 
yourself 10 minutes for this part of the survey* Write your answer below; 



1* Discuss the success and/or failures of at least 3 reform groups in - 
the Aiiiarican experience. 
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List 3 technological Innovations in ^erlcan cultuxe and explain 
during the time allotted how these changes have affected \.ne social 
life of Individuals and/or groups. 



Mr. Nixon was a severe critic of wcrld CommuKilsm as a United States 
Representative. As President, he has traveled to Moscow and Peking. 
Has the change In attitude come about d^e to his change of political 
roles? Explain. 




Discuss the separation of powers doctrine and the checks and balances 
system. Give examples to support your statements. ^ 



Discuss the significance of frontiers In American history (Indian, 
mining, cattle, railroad development, etc.). Include an explanation 
of the "safety valve" theory and the Turner Thesis. 



■ ' USE OF HATIONAL ASSESSMENT MODEL FOR CLASSROOM EVALUATION 

■ A Paper Presented to the 
H Regional Conference 
I of the 

■ National Council for the Social Studies 
B Virginia Beach, Virginia 

■ April 24-26, 1975 

I By Richard fiulsart 

B National Assessment of Educational Progress 

I Last minute cramming and comparisons of test scores are fairly common 

H student reactions to individual achievement testing in the classroom. 

H While the diagnostic value of the individual achievement test may well 

■ be worth the anxiety it causes. National Assessment offers a model for 
H evaluation that can be used for diagnostic purposes without the student 
I having to fear failure or having his ability unfairly judged by peers. 

I ^National Assessment exercises are administered on a national basis to 

I groups of young people ages 9, 13, 17 and adults ages 26*35. Although 

I each person in the National Assessment s^^ple completes an exercise book^ 

I let, the results are analyzed in tenos of group rather than individual 

H achievement. Due to its large sample. National Assessment is able to 

■ provide group achievement results for the nation as a whole, geographic 

■ regions of the- country, ages, sex, colors, sizes and types of community 
I { and levels of parental education. The teacher with a relatively small 

■ sample can perhaps only analyze group results on the dimension of the 
I entire class; however, this data can be very valuable in providing the 
I teacher with information about the understandings of the students and 
B his or her own teaching effectiveness.^ In order to illustrate the use 
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of the National Assc^ssment model, I hd^e chosen to discuss three exercises 
reported by National Assessment.^ 

The following exercises presented In National Assessment's recent 
report, Political Knwledge and Attitudes^ are two of a number of exer* 
clses developed to measure young people's knowledge of and attitudes 
toward constitutional rights. 

The first Is a multiple-choice exercise concerned with measuring the 
knowledge of where a person would find information about one's civil 
rights. 



If a citizen of the United States wants to find a 
statement of his civil rights, in which one of the 
following should he look? 

CZ:) In the Bible 

CZ:) In the Constitution 

CZ:) In the Articles of Confederation 

CZ^ In the Declaration of Independence 

CZ5 I don' t know. 



For classroom use, such an exercise can be administered to the stu- 
dents as a group and the results tabulated for the entire class. Then 
both the students and the tescher can examine the results in terms of the 
class's awareness of the conti^nt of che documents mentioned in the exercise 



Exercises and assessment results in the areas of citizenship, literature, 
music, reading, science, social studies and writing can be obtained for 
a minimal fee upon request from the Superintendent of Documents, United 
States Government Printing Office, Wsshlngton, D.C. 20402. Objectives 
are also available for all lO areas assessed by National Assessment— art, 
career and occupational development, citizenship, literature, mathematics, 
music, reading, science, social studies and writing. 




23 



The class results can also be conpAred with Che following nationwide results 
reported by National Asressment : 



Choices 

In th^ Bible 
*In the Constitution 
In the Articles of Cot 
In the Declaration of 

Independence 
I don^t know 
No rer^^onse 

*Correct response 



A^e 13 Age 17 Adult 



IX 


OX 


IX 


63 


84 


78 


11 


6 


4 


20 


8 


14 


5 


2 


4 


0 


0 


0 



Such comparisons can lead to a discussion of the purposes of the documents 
as well as some decision on the part of both the teacher and the students 
regarding the need for further work on sources of information ibout civil 
rights. 

The second exercise calls for a written response to a question which 
was designed to ineasure the objective: believe in the freedoms of the 
First Amendment and can justify their beliefs^ 
Part A 

Should a newspaper or magazine be allowed to publish 
something that criticizes an elected government 
official? 

<Z:> Yes 

O No 

<Z> Undecided 
Part B 

Please give a reason for any answer you selected. 
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Open-ended exercises such as the one above can be scored by an entire 
class. The responses given by the class to this exercise can be recorded 
and handed out to the students. The students can decide on the general 
types of categories of answers that are being given and then see if they 
can fit all the responses^ under these categories. There can be a discus- 
sion about the categories in terms of which categories indicate an accept*-" 
able achievement of the objective the exercise was designed to itieasure. 
The very process of deciding on categories and their acceptability should 
create considerable classroom dialogue. Once the categories are decided 
upon"^ the results can be tabulated (i.e., the number of students able to 
give an acceptable response to all or part of the exercise). The results 
of the class performance on the exercise can also be compared with the 
National Assessment results. 

The National Assessment reports show major types of responses 
(categories) and the percentages of the nationwide sample falling in each 
category* Following are the national results for the exercise on "Should 
a newspaper or magazine be allowed to publish something that criticizes 
an elected government official?" 

Part A 

Choices Age 13 Age 17 Adult 



Yes 


49% 


732 


81% 


No 


38 


16 


8 


Undecided 


13 


11 


11 


No response 


0 


0 


0 
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MAJOR CATEGORIES OF RESPONSES 
Acceptable Achievement of Objective 

Age 13 Aae 17 Adult 

Freedom of the press; constitu- 



^tlonal rights 




22% 


30Z 


n@|d to be Infonaed and/or 








'Resulting citizen actions 


13 


25 


21 


R^ht to cay what they*feel 


5 


7 


5 


Help the official; help the 








country 


2 


4 


6 


Conditional; if it is true 


3 


7 


6 


Conditional with respect to 








libel (limits freedom of press) 


0 


1 


1 


Other acceptable 


1 


2 


4 


/ 

Unacceptable Achievement of Objective 




ii^isn*t fair; may hurt the 








official 


13 


6 


2 


May publish statements that are 








not true 


3 


2 


1 


Paper should not criticize — no 








reason given 


4 


1 


1 


Would destroy people's confidence 








in government; start riots 


2 


1 


0 


Conditional; other conditions 


2 


6 


9 


Vague, nonsensical, other 






* 


''unacceptable responses 
I don't know 


22 


12 


8 


7 


3 


3 


Ho response 


5 


2 


3 



A con^l^lson if these national results with the responses of the class 
can lead to discussions concerning both the differences In types of cate* 
gorles as well as differences In the percentages falling In each category. 
There can also be considerable debate over National Assessment s judgment 
as ^ categories which are acceptable or unacceptable achievements of the 
objective .^'^^''T^jcl-ass may have different conceptions regarding acceptable 
responses y and it would be Interesting to explore the criteria of the 
class for judgments concerning categories. 
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National Assessment reports also provide results for groupings 
besides the nation as a whole. For example^ in the report Political 
Knowledge and Attitudes, there are charts comparing assessment results by 
region of the country, sex and color of respondents, size and type of 
community and education of parents. These results are reported for combina^ 
tions of exercises under a topic or theme such as "Knowledge of and Atti" 
tudes Toward Constitutional Rights." Most Assessment^ reports provide 
results for these reporting groupings for each exercise in the sppendices. 
The class will probably find considerable value in examining the. variations 
in responses according to region^ sex^ color^ type of conmninity and parental 
background. 

Not all of National. Assessment's exercises are paper-and<*pencil types 
of measures. Included in each of the areas assessed are performance type 
instruments. The following paragraph describes a group task exercise 
administered in the 1969 assessment of citizenship. This particular exer- 
cise was used to measure the objective ''Apply democratic procedures on a 
practical level when working in a group." 



A group of eight students were asked to choose from a 
list of 12 issues (e.g. ^ censorship^ curfew^ home 
duties* adult^''^v|^ and books* school assignments) 
the 5 most important issues between teenagers and 
adults. These eight students were then asked to rank 
the 5 issues in order of importance and to write a 
recoimnendation for at least the 2 most important 
problems or for all 5 problems if they had time. They 
had 30 minutes to complete the task. The only nile was 
that a majority of the group had to agree on anything 
they wrote. Two observers Recorded individual acte of 
group members as they discUssed the issues, each 
observer recording different types of behaviors. At 
no time did the observers participate in the discussions. 



In the case of National Assessment's administration of this exercise* 
the observation and recording of behaviors was handled by professionally 
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trained observers. In the case of classroom use» It would be very appro^ 
prlate to have students do the observation and recording. 4|g^dents ^ ^ 
selected to observe the group*s behavior can use a form s^n^r^ to the 
sample in Table I to record the frequencies of behavior* or the class c^I 
establish its own criteria and develop its own form for recording group ^ 
behavior. 

Following the administration of this exercise* tbe^ w^^ten recommenda- 
tions should be evaluated by the class to determine the effectiveness of 
the group^s efforts in accomplishing the task. The frequencies for each 
type of behavior can be tabulated and used to determine the extent of indi- 
vidual participation and ways of improving group interaction. The number 
of the students interacting can be varied and issues for the task changed 
according to the subject for study or the interests of the students. 

^From classroom discussions of the various exercises, the teacher 
should be able to obtain clues regarding -the undersnndings and attitudes' : 
of the students. The teacher may "want to sharpen h^ or her evaluation b^* 
giving such exercises at the beginning of the year or be^hre a unit on th^^ 
subject covered by the exercises and then again at various points during^^- 
the year or the unit.y A record a£ the changes in understandings or at^^^ 
tudes over time can provide some interesting in£t>rmatio^^f§^arding the 
needs of students and the effectiveness of the instruction. The 'teacher 
may want to estimate the degree of achievement expected students r: 

before the exercises are administered. The comparison of the teacher^s 
expectations and the actual scores of the class should ^Iso provide the 
teacher with an excellent point of reference for further work. 
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TABLE 1 

RECORD OF FREQUENCE OF BEHAVIOR 
Check Each Tiioe Behavior Is Observed 



States any reason 
for and/or against 
some position 
related to the con- 
tent of the group* s 
task 


Makes a clear 
statement of posi- 
tion on some issue 
relevant to group 
recommendation 


0-Yields to opposition for 
sake of group progress 

Y-Convinces majority or 
group to accept view- 
point 

C'^Makes a clear stateioent 
in opposition to an 
opinion put forth by two 
members of the group 


1. 






2. 






3. 






4. 






5. 






6. 






7. 






8. 







Of 



CO 



Attempts to 
organize the 
task for the 
group or attempts 
to change some 
procedure for 
accomplishing 
task 


Seeks informa* 
tion, asks opin- 
ion, seeks 
clarification 
from other group 
members 


Defends other 
group members 
the right to 
speak out or 
have a different 
opinion 


Makes comment 
unrelated to 
task, or refusej 
to cooperate 


1. 








2* 








3. 








4. 








5* 








6* 








7* 








8* 









Of 

D 
3 



C 
Of 

D 
4J 

CO 



3{ 
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The exercises such as those developed by National Assessment can also 
be constructed by teachers and students working together cooperatively. 
Series of exercises measuring a particular objective can be developed so 
that the teachers and student can examine a range of responses on a par- 
ticular Subjects On the basis of class experience with this type of 
evaluation, students can explore the subject of survey sampling, and, with 
exercises that they have constructed, conduct their owii surveys In their 
school and out In the community^ Evaluation can be both a constructive 
and nonthreatening process In the classroom If the development of the Instru* 
ment as well as Interpretation of the results are shared by the teachers 
and students. 



Results reports for assessments In eight 
learning areas can be ordi&red through the 
Superintendent of Documented Government 
Prin ting Offica^ Washington^ B. 20402 . 

For complete publications list with prices 
and stock numbers, as well as materials 
available through K&tional Assessment with* 
out charge, write to NAEP Publications, 
address below^ 



NATIONAL ASSESSMENT OF EDUCATIONAL PROGRESS 
A Project of the Education Commission of the States 
I860 Lincoln Street* Suite 700 
Denver* Colorado 80203 
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